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Abstract — In this papeiQ we consider the fc-set agreement 
problem in distributed message-passing systems using a round- 
based approach: Both synchrony of communication and failures 
are captured just by means of the messages that arrive within 
a round, resulting in round-by-round communication graphs 
that can be characterized by simple communication predicates. 
We introduce the weak communication predicate "Psrcs(fc) and 
show that it is tight for fc-set agreement, in the following sense: 
We (i) prove that there is no algorithm for solving (fc— l)-set 
agreement in systems characterized by "Psrcs(fc), and (ii) present a 
novel distributed algorithm that achieves fc-set agreement in runs 
where "Psrcs(fc) holds. Our algorithm uses local approximations of 
the stable skeleton graph, which reflects the underlying perpetual 
synchrony of a run. We prove that this approximation is correct 
in all runs, regardless of the communication predicate, and show 
that graph-theoretic properties of the stable skeleton graph can 
be used to solve fc-set agreement if "Psrcs(fc) holds. 

I. Introduction 

The quest of finding minimal synchrony requirements for 
circumventing the impossibility of distributed agreement prob- 
lems like consensus [9| has always been a very active research 
topic in distributed computing. Since the exact solvability 
border of consensus has been researched exhaustively, see e.g., 
10, 0, lfl2l . the attention has shifted to weaker agreement 
problems, in particular, fc-set agreement (TJ, iflTl . Ifl4l . which 
allows the processes in a distributed system to agree on at most 
fc different values. For fc > 1, the problem itself is possibly 
not as interesting as consensus (fc = 1) from a practical point 
of view, except for partitionable systems that need to reach 
consensus in every partition. In any case, fc-set agreement is 
highly relevant from a theoretical perspective, as it allows to 
study what level of agreement can be achieved in a fault- 
tolerant distributed system. This question is definitely relevant 
in practice, e.g., for name-space reduction (renaming) and 
similar problems. 

One way to model synchrony requirements is through the 
use of round models. Round-based distributed algorithms 
execute in a sequence of communication-closed rounds, which 
consist of message exchanges and processing steps. The 
classic partially synchronous models of Dwork et. al. Q 
were probably the first to allow some messages not to arrive 
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within a round due to asynchrony (i.e., non-timeliness), rather 
than solely due to failures. The seminal work by Santoro 
and Widmayer |fT31 , 11161 unified the treatment of asynchrony 
and failures by considering synchronous processes that only 
suffer from "end-to-end communication failures". This idea 
also underlies the Round-by -Round failure detector (RRFD) 
approach by Gafni [ 10 1, which assumes a local RRFD that tells 
whether a process shall wait for a round message from some 
other process or not. The actual reason why a receiver process 
does not get a message from the sender process is considered 
irrelevant here. The Heard-Of (HO) model ||3], H integrates 
this unified treatment of failures and asynchrony of [15], lfl6l 
with a flexible way of describing guarantees about commu- 
nication. The basic entity of this model are communication- 
closed rounds and HO predicates, which specify conditions on 
the collection of heard-of sets: For each round r and process 
p, HO(p,r) denotes the set of processes that p hears of (i.e., 
receives a message from) in round r. 

In this paper, we will use properties of communication 
graphs for studying fc-set agreement in message passing 
systems with very weak synchrony requirements. In fc-set 
agreement, correct processes must output a single value based 
on values proposed locally, with no more than fc different 
values being output system-wide. 

Detailed contributions: We introduce an algorithm for 
fc-set agreement, which exploits a natural correspondence be- 
tween communication predicates and round-by-round "timely 
communication" graphs Q r in a run; Q r contains an edge 
(q p) when process p hears of q in round r. Our 
algorithm incorporates a generic method for approximating 
the stable skeleton Q noc , which is the intersection of all Q T 
and reflects the underlying perpetual synchrony of a run. We 
also introduce the class of communication predicates 7- > srcs (fc), 
which guarantees that at least two processes in every subset of 
fc + 1 processes hear from a common process, in every round. 
Using the graph-theoretic properties of Q n °° guaranteed by 
the predicate V SIC s(k), we show that our algorithm solves fc- 
set agreement in all runs where V SIC s(k) holds. Moreover, we 
also show that V SIC!i (k) is "tight" for fc-set agreement, as it is 
too weak for solving fc — 1-set agreement. 



II. Computing Model and Problem Definition 

We consider distributed computations of a set of processes 
II communicating by message passing. Moreover, we consider 
that the computation is organized in an infinite sequence of 
communication-closed |8| rounds; that is, any message sent in 
a round can be received only in that round. As in the models 
of GafniflOl and Charron-Bost and Schiper [4|, we will 
express assumptions about the synchrony and the reliability of 
communication in a system by a predicate that characterizes 
the set of edges in the communication graph of each round. 
Intuitively speaking, there is an edge from process p to q in 
the communication graph of round r is q received p's round r 
message. We will in fact name a system by its predicate, that 
is, in a system V the collections of communication graphs 
of each run of an algorithm in that system will must fulfill 
predicate V '. 

We now formally define computations in our round model. 
As in the aforementioned models, an algorithm is composed 
of two functions: The sending function determines, for each 
process p and round r > 0, the message p broadcasts in round 
r based on the p's state at the beginning of round r. The 
transition function determines, for each p and round r and the 
vector of messages received in r, the state at the end of round 
r, i.e., at the beginning of round r + 1. Clearly, a run of an 
algorithm is completely determined by the initial states of the 
processes and the sequence of communication graphs. 

For each round r, we denote the communication graph by 
Q r = (V,E r ), where each node of the set V is associated 
with one process from n, and where E r is the set of directed 
timely edges for round r. There is an edge from p to q, denoted 
as (p —> q), if and only if q receives p's round r message 
(in round r)@ To simplify the presentation, we will denote a 
process and the associated node in the communication graph 
by the same symbols. However, as we differentiate between V 
and II, we will always be able to resolve possible ambiguities 
by stating from which set a node or process is taken. We will 
write p e Q r and (p q) g Q r instead of p e V resp. 

{ P ^q)eE\ 

We are primarily interested in the round r skeleton Q n r of 
Q r , which we define as the subgraph consisting of the edges 
that have been timely in all rounds up to round r. Formally, 
G nr := (V,E nr ) where E nr := f)o<r'^r Er - The cmcial 
property of E nr is that once an edge is untimely in some 
round r, it cannot be in Q n r , for any r' ^ r. That is, Vr > 
0: E nr D E nr+1 , which implies the subgraph relation 

Vr > 0: G nr D G nr+1 . (1) 

We are particularly interested in the stable skeleton of a run, 
which we define as the intersectior0 over all rounds, i.e., 

£ no ° : =n re M+£ nr - ( 2 ) 

Considering that a run a consists of infinitely many rounds, 
whereas our system consists of only a finite number of 

2 Since we consider communication-closed rounds, a message sent in round 
r cannot be received in any later round. 

3 For simplicity, we set G P G' := {V H V , E D E'). 



processes, it follows that the number of possible distinct stable 
skeletons must also be finite. Consequently, the subgraph 
property ([T) implies that there is some round rsT when Q n °° 
has stabilized, i.e., Vr ^ r ST : G nr = G noc - 

As mentioned in the introduction, our algorithm will solve 
fc-set agreement by approximating the stable skeleton of a 
run. The first step in this effort is to use the locally avail- 
able information about the communication graph, which is 
captured by the notion of timely neighbourhoods. The timely 
neighborhood of p, denoted as PT(p, r), is the set of processes 
that process p has perceived as perpetually timely until round 
r. In other words, p has received a message from every 
process in PT(j>, r) in every round up to and including r, 
i.e., PT(p,r) := {q \ (q— > p) € G nr }. Analogously to (Q} 
and (0, we have 

PT{p,r) D PT(p,r+l) (3) 

and define 

PT(p):=p\PT(p,r). (4) 

We will make heavy use of the standard graph-theoretic 
notion of a strongly connected component of Q nr . Note that 
we implicitly assume that strongly connected components 
are always nonempty and maximal. We use the superscript 
notation C when talking about a strongly connected compo- 
nent of G nr . Moreover, we write C r v to denote the (unique) 
strongly connected component of Q n r that contains process p 
in round r. The strongly connected component C g nco 
that contains p in a run is defined analogously to (O as 

qr : = n c;. 

Note that when p and q are strongly connected in Q n r , then 
they are also strongly connected in all Q nr , for < r' ^ r. 
From property ([T) of Q n r , we immediately have 

Vr > 0: C; 3C; +1 . (5) 

We will also use directed paths in Q nr , where we assume that 
all nodes on a path are distinct. 

Let C r C Q nr be a strongly connected component. If C 
has no incoming edges from any q G Q nr \ C r , we say C r is 
a root component in round r. Formally, 

Vp S C r Vq e G nr : (q -)• p) £ G nr => q £ C r . 

Figure [TBI shows a graph with 2 root components {p3,P4,P5} 
and {pi,p 2 }- 

Regarding the relation to the existing round-by-round mod- 
els, we shortly recall what their predicates are based on: In 
the Heard-Of model [|4], for each round r and each process p, 
the set HO(p,r) contains those processes that p hears from, 
i.e., receives a message from, in round r. In the case of the 
Round-by-Round Fault Detectors [10|, the output of p's fault 
detector in round r is referred to by D(p, r). In each round r, 
process p waits until it receives a message from every process 



that is not contained in D(p, r). While it is possible thatp also 
receives a round r message from a process in D(p, r), we will 
consider that this is never the case. From this it is evident that 
we have the following correspondence between our skeleton 
graphs and the HO/RbR model: 

Vr'^rpe HO{q,r r ) 
Vr' < r: p g D(q,r') 



(6) 



Thus a process can determine its timely neighbourhood in 
the two models as follows: 



PT(p,r) = 



'n 

n\(U 



0<r'<r 



HO(p,r>) 



■>0<r'^r 



D(p,r') 



(7) 



As in the HO-model, we model a crashed processes by 
an "internally correct" process that no other process receives 
messages from after it has crashed ||4] Sec. 2.2]. This mod- 
elling allows us to require that all processes decide. For a 
more detailed discussion on the relation between models where 
crashed processes actually stop and the HO-model, we refer 

to in. 

A. k-Set Agreement 

The k-set agreement problem was introduced in [5|. Every 
process p starts with a proposal value v and must eventually 
and irrevocably decide on some value adhering to the follow- 
ing three constraints: 

fc-Agreement: Processes must decide on at most k different 
values. 

Validity: If a process decides on v, then v was pro- 
posed by some process. 

Termination: Every process must eventually decide. 

Note that the k-set agreement problem was shown to be im- 
possible in the asynchronous system model (see (TJ, ifTTl . [14|) 
if / ^ k processes can crash. Recalling the correspondence 
between crashed processes and process that no one hears of, 
it is not surprising that this impossibility also holds for the 
system P lms :: TRUE, where all runs are admissible. 

III. A Tight Communication Predicate for fc-SET 
Agreement 

In this section, we introduce a predicate that, together with 
Algorithm Q] in Section [IV] is sufficient for solving fc-set 
agreement. 

For a run a, predicate V STC $(k) requires that in every set S 
of k + 1 processes, there are two processes q, q' that receive 
timely messages from the same common process p, in every 
round. We say that p is a 2-source and q, q' are timely receivers 
of p in a. 

7>«(p, S) :: 3q,q' eS,q^q': P G (PT(q) D PT(q')) 
P SICS (k) :: VS, |5| = k + 1 Bp G II: P src (p, S) (8) 

Note that p is not required to be distinct from q and q': 
Psrcs(k) still holds if p = q, i.e., p always perceives itself 
in a timely fashion. Regarding communication graphs, this 
predicate ensures that any induced sub-graph S of Q n °° with 



k + 1 nodes contains distinct nodes q and q', such that, for 
some node p, edges (p q) and (p — > q') exist (one of which 
may be a self-loop). Figure [Tb] shows the stable skeleton graph 
in a run where P SICS (k) holds for k = 3. 

At a first glance, it might appear that the perpetual nature 
of 'Psrcs(fc) is an unnecessarily strong restriction. To see why 
some (possibly weak) perpetual synchrony is necessary, con- 
sider the predicate ()P sr cs(k) that satisfies (JHJ just eventually, 
and suppose that there is an algorithm A that solves k- 
set agreement in system {}P srC s{k). Due to its "eventual" 
nature, < 07- > srC s(fc) allows runs where every process forms a root 
component by itself, i.e., hears from no other process, for a 
finite number of rounds. Moreover, for any k, the (infinite) run, 
where a single process forms a root component forever and 
thus has to decide on its own input value, is admissible. Using 
a simple indistinguishability argument, it is easy to show that 
processes decide on n different values. 

The following result will be instrumental in Section [IV] 
where we show how to solve fc-set agreement with P srC s(k). 
Note that Theorem Q] is independent of the algorithm em- 
ployed. 

Theorem 1: There are at most k root components in any 
run that is admissible in system P STC s(k). 

Proof: Assume by contradiction that there is a run a of 
some algorithm A that is admissible in system P srC s{k), where 
there is a set of £ k + 1 disjoint root components R = 

, . . . , } containing processes pi, . . . , Pk+i , ■ ■ ■ ,Pt- Let 
r be the round where every strongly connected root component 
C~ G R has stabilized, i.e., Vi: C r p% = C™. That is, any two 
distinct root components in R must already be disjoint from 
round r on. Since a satisfies 7 ? srcs (A;) and £ ^ fc+1, there must 
be a 2-source p such that, for two distinct processes Pi,pj G 
{pi, . . . ,p k +i}, it holds thatp G (PT(pi) n PT( P] )) . By ©, 
it follows that the edges = (p — >• pi) and e 3 = (p — > pj) are 
in Q n r . Considering that C£. and C£. are root components by 
assumption, i.e., do not have incoming edges, it must be that 
e,; G Cp . and ej G C£ . , and therefore p G C r p . n C£ . . This, 
however, contradicts the fact that Cp. and Cp. are disjoint, 
which completes our proof. ■ 

A. Impossibility of (k~l)-Set Agreement 

We will now show that P sms (k) does not allow to solve 
[k— l)-set agreement. More specifically, we will prove this 
by assuming the existence of such an algorithm A, and then 
construct a run fulfilling P SICS (k) where processes decide on 
k (instead of k — 1) different values. 

Theorem 2: Consider any k such that 1 < k < n. There 
is no algorithm A that solves (k— l)-set agreement in system 
P sl -c S (k). 

Proof: Assume for the sake of a contradiction that such 
an algorithm A exists. Suppose that all processes start with 
pairwise distinct input values. Consider the run a and a fixed 
set L of k — 1 processes that only hear from themselves, 
formally speaking, Vp G L: PT(p) = {p} . Moreover, there 



is one process s such that every process not in L only hears 
from itself and s, i.e., 

Mp £ II\L: PT{p) = {p,s}. 

Since, by validity and termination, processes eventually have 
to decide on some input value and processes in LU{s} cannot 
learn any other process' input value, they have to decide on 
their own value. Thus, we have k different decision values, as 
we have assumed a unique input value for each process, and 
therefore a violation of (k-l)-agreement. 

What remains to be shown is that this run a actually fulfills 
^srcs(fc)- Recall equation (0, i.e., the definition of P S[CS , and 
consider for any set S of size k + 1 the set P = S \ L. Since 
IS* \ L\ ^ 2, the set P contains at least two distinct processes 
that permanently hear from s (one of which may be s). That 
is, process s is the required 2-source for any set S of k + 1 
processes. ■ 

IV. Approximating the Stable Skeleton Graph and 
Solving /c-Set Agreement 

In this section, we present and analyze an algorithm that 
solves k-set agreement with predicate P srcs (fc). Algorithm Q] 
employs a generic approximation of the stable skeleton graph 
of the run, which works as follows: 

First, every process p keeps track of the processes it has 
perceived as timely until round r in the set PT p , updated in 
Line|9] Lemma[3]will show that PT p satisfies the definition of 
PT(p, r), for all rounds r. In addition, every process p locally 
maintains an approximation graph G p of the stable skeleton, 
denoted G r p for round r, which is broadcast in every round. 
If a process q receives such a graph G p from some process p 
in its timely neighborhood PT(q,r), it adds the information 
contained in G p to its own local approximation G r q . Note that, 
in contrast to the stable skeleton graph Q n r , the approximation 
graph G p is actually a weighted directed graph. The edge 
labels of G p correspond to the round number when a particular 
edge was added by some process, i.e., the edge (q' A q) is in 
G p if, and only if, q' £ PT(q, r) (cf. Lemma[3jb)). To prevent 
outdated information from remaining in the approximation 
graph permanently, every process p purges all edges in G r p that 
were initially added more than n— 1 rounds ago. Figures [TcflTh] 
show this approximation mechanism at work. 

For fc-set agreement, process p only considers proposal 
values for its estimated decision value x p that were sent by 
processes in its current timely neighborhood, i.e., in PT P . This 
ensures that p and q will have a common estimated decision 
value Xp — - in round n, if they are in the same strongly 
connected component (cf. Lemma [T4V To determine when to 
terminate, p analyzes its approximation graph in every round 
r ^ n and decides if G p is a strongly connected graph. 

Why is this decision safe with respect to the agreement 
property? Using our graph approximation results, we will show 
in Lemma[T5lthat any strongly connected approximation graph 
contains at least one root component in the stable skeleton 
graph. Furthermore, if two processes decide on different 



Algorithm 1 Approximating the stable skeleton graph and 
solving k-set agreement with V srC s{k) 

Variables and Initialization: 

i: PT p G 2 n initially II 

2: x p G N initially v p II Estimated decision value 

3: Gp :— (Vp,Ep) initially {{p} ,0) II weighted digraph 

4: decidedp G {0, 1} initially II is 1 iff p has decided 

Round r: sending function Sp\ 
5: if decidedp = 1 then 
6: send (decide, x p ,Gp) to all processes 
7: else 

a: send (prop,x p ,Gp) to all processes 

Round r: transition function T£ : 
9: update PTp 

io: if received (decide, x q ,_) from q G PT P and decidedp — 
then 

11: 'Ju <p ^ 30 q 

12: decide on x p 
i3: decidedp <— 1 

i4: II Approximate stable skeleton graph: 

15: Gp^ <{ P },0> 

is: for q G PTp do 

n: add directed edge (q A p) to E v 

18: Vp^VpUVq 

is: for every pair of nodes (pi,Pj) G V p x V p do 
20: Rij <- {r e I 3q G PT P : (p t ^ p 3 ) G E q } 
21: if // • U then 
22: r ma x <— max(Rij) 

Ep <- E p U{( Pl r ^ Pj)} 

24: discard all (p; -4 P j) from E v where r e ^ r — n 

25: discard pi ^ p from V p if p is unreachable from pi in G p 

26: if decidedp = then 

27: x p <— min{a; ? | q G PT P } 

28: if r ^ n and G p is strongly connected then 

29: decide on x v 

30: decidedp <s— 1 



values, it follows that their approximated graphs in the rounds 
of their respective decision are disjoint. Since Theorem [T] 
confirms that there are at most k root components in any run 
where V STCS (k) holds, there can be in fact at most k different 
decision values. 

A. Approximation of the Stable Skeleton Graph 

Throughout our analysis, we denote the value of variable 
var of process p at the end of round r as var p . When we use 
the subgraph relation (C) between graphs C p and G p , we mean 
the standard subgraph relation between C p and the unweighted 
version of G p . We first state some obvious facts that follow 
directly from the code of the algorithm: 

Observation 1: For any round r > it holds that p G G p 
and that no edge (q' —> q) 6 G p has s ^ r — n. 

Note that, after the initial assignment, p only updates 
variable PT p in Line [9] which is equivalent to (0. From 
this and the inspection of Lines [15] and [17] Lemma [3] follows 
immediately: 




Fig. 1: A system of 6 processes where (3) holds. The stable skeleton graph for round 2 is depicted in Figure [Tal [TBI shows 
the stable skeleton graph for the entire run. For simplicity, we omit self-loops, i.e., Vpi : p, G PT(pi). Figures ITcVThl show 
process pg's approximation of g n °° during rounds 1 to 6. 



Lemma 3: It holds that q G PT{p, r) if, and only if, all of 
the following are true: 

(a) q e PT£, 

(b) p adds a directed edge q — > p to G p by executing Line [171 
in round r, and 

(c) for any r' ^ r, there is no other edge q A p in G p . 
The following lemma shows that the approximation graph 

Gpe+i accurately reflects the timely neighborhood of a process. 
That is, if p\ is connected to pe+i through a path of length 
£, then pi+i will add the timely neighborhood information of 
pi to its approximated graph by round I. 

Lemma 4: Suppose that there exists a directed path 

r = (pi ->•... ->■ p^+i) 

in C* n r for round r n, where T has length £ ^ n — 1. Then, 
V<? 6 PT(pi, r - I) it holds that 

(a) edge (q -4 pi) is in G pe+1 where r r 9 r — £, and 

(b) G£ contains no other edges from q to pi. 

Proof: Consider an arbitrary g G PT(pi,r—£). The proof 
proceeds by induction over the edges of path V indexed by k. 
That is, we show that for all k, with ^ k ^ £, it holds that 
there is an edge e = (q -4 pi) in G p ~^+ k where r — £ + k ^ 
r k ^r -I. 

For the base case (fc = 0), we have to show that the edge e 
is in Gp~^, but this already follows from q G PT(pi,r — £), 
by Lemma [3] 

For the induction step, we assume that the statement holds 
for some k < £ and then show that it holds for k + 1 as well. 
In round r — £ + (fc + 1) process pi+fc broadcasts its current 
graph estimate, i.e., G p ^^ k to all. We know that Pi+(fc+i) 
will receive this message since (pi+fc — > Pi+(fc+i)) is in the 
path r C g nr , which means that 

Pl+k G PT{ Pl+(k+1) ,r -£+(k + 1)). 



By the induction hypothesis, the edge (q -4- pi) is in G p ~^~ k 
and therefore will be among the edges that Pi+(fc+i) considers 
in Line|20] This in turn implies that Pi+( k +i) will add an edge 
q ''^ Pi to its graph Gp^^j^ 1 ^ in Line [23] whereby r^+i 
is calculated in Line [22] such that r k+ i ^ r k . Moreover, by 
induction hypothesis we have r k ^ r — £ > r — n, which 
ensures that the edge will not be discarded in Line [24] Since 
the code following the for-loop in Line [19] is executed exactly 
once for every edge, no other edge q ^ p\ is added to 
G Pl+{k+l) . This completes the proof our lemma. ■ 

The next lemma shows that the approximation graph of 
correctly (over)estimates the strongly connected component 
from round n on: 

Lemma 5: Let r > n and consider the strongly connected 
component C p containing p in Q nr . Then, it holds that G p 3 

nr 

v 

Proof: Consider any edge (q' —> q) G C p . Since C p is 
strongly connected, there is a directed path between any pair of 
processes in C' p , in particular there is a path of length £ ^ n — 1 
from q to p. By the definition of C p we know that q always 
perceives q' as timely in all rounds up to round r, which means 
that q 1 G PT(q, r — £). Then, by applying Lemma [4] we get 

that the edge (q' -4 q) is in G p , for some r' Ss r — £, which 
shows that C p is a subgraph of G p . ■ 

Lemma[3]showed that the timely neighborhood is eventually 
in the approximated graph. We now show that our approxima- 
tion contains only valid information: 

Lemma 6: Let r ^ 1 and suppose that there is an edge 
e = (q' A q) in the approximated stable skeleton graph G p 
of process p. Then it holds that q' G PT(q, s). 

Proof: Note that processes only add edges to their ap- 
proximation graphs in Line [17] or in Line [23] If an edge is 
added via Line [23] then this edge has previously been added 



by another process by executing Line \T7\ Therefore, every 
edge must have been added by some process via Line [17] In 
case of e, this process can only be q. By Lemma[3]this happens 
in round s and q' G PT(q, s). ■ 
The following Lemma [7] is in some sense the converse result 
of Lemma [5] as it states that the approximated graph must 
approach C p from below, if it is strongly connected: 

Lemma 7: Let r ^ 1 and consider the strongly connected 
component C p , If the approximated skeleton graph Gp +n_1 is 
strongly connected, then C p 3 G 7 p +n ^ 1 . 
Proof: Consider any edge 

e = (q' ^ q) G G£ +n_1 . 

By Lemma [6] we know that q' 6 PT(q, r'). It follows by 
the subset property ((3) that g' G PT(q, r), as Observation Q] 
implies 

r > (r + n — 1) — n = r — 1. 

Therefore, there is an edge (g' — > q) in C? nr . 

It follows that G p +n ~ 1 is isomorphic to a (not necessarily 
maximal) strongly connected component S r in Q nr . Because 
Cp and S r both contain p, their intersection is nonempty, i.e., 
C r p 3 Gp +n_1 . " ■ 

As a final result about the approximated skeleton graph, we 
show that once the approximation G p is strongly connected in 
round r ^ n, it is closed w.r.t. strongly connected components. 
This means that G p can be partitioned into disjoint strongly 
connected components in g n °°. 

Theorem 8: Suppose that R ^ n. If the approximated 
skeleton graph is strongly connected, then it contains the 
strongly connected component of every q G G p . 

Proof: Consider any q G G p and its strongly connected 
component C£°. From (0 and Lemma [7] it follows that 

Q £ Gp Q C«-" +1 C <* 

i.e., ij e flCj. Moreover, due to the well-known fact that 
two maximal strongly connected components in a digraph are 
either disjoint or equivalent, we get that C q = CL 

Now suppose the theorem does not hold. Then there exists 
some q' G such that q' $ G p . Due to Lemma [3] q' cannot 
be contained in Cf, but due to ©, q' G C q D C£°. Therefore, 
± C*, and thus Cf n = 0. Since g} is strongly 
connected and contains q, it also contains a path 

r = {q = Pe ~> > Po = p), 

such that 

Vi, 0<i<*:p i+ i ePT{ Pl ,R-i). 

Let j be the minimal index i such that pj G C^, and let 
Tj = (pj —>••••—> po) be the path remaining from pj. 

As both q' and pj are in C^, there is a path V in C^. Let 
fc be the length of this path. Moreover, by applying Lemma |4] 
we get that G^7^ contains the outgoing edge e of q' on this 
path, labeled with some round 

r'^R-j- k. (9) 



But then, by the definition of T, it follows that when G p 
contains pj — which it does — then it must also contain q' , 
unless some process pi (i < j) removed e from its set of 
edges in line [24] in round R — i because r' ^ R — i — n. Since 
round R at process p(= po) is the latest round when this can 
occur, we get that r' ^ R — n, and thus, by (O, 

R-j-k^r'^R-n, i.e., j + k ^ n. (10) 

Let A be the subgraph obtained by concatenating paths V 
and Tj. By construction, Tj and V only share node pj, and 
thus A is a (simple) path and must have length j+k ^ n—1, as 
no path can exceed length This contradicts (TTOb and thus 
completes the proof that q 1 is in G p . The proof showing that 
all edges of are in G p proceeds analogously, by assuming 
that some edge in C£° ending in q 1 is not in G p . ■ 

B. k-Set Agreement 

In this section, we will show that Algorithm [T] not only 
approximates the stable skeleton graph, but also solves fe-set 
agreement. Our previous results allow us to immediately prove 
the validity and the termination properties. 

Lemma 9 (Validity): If a process decides on v, then v was 
the initial value of some process. 

Proof: Observe that the decision value x p of any process p 
is initially set to its proposal value v p , which is then broadcast. 
On all subsequent updates of x p in Line [27] a value x q that 
was sent by some process q (which originated from some v q > ) 
is assigned, therefore validity holds. ■ 

Lemma 10: Every process decides at most once in any run. 
Proof: Observe that no process executes Line [29] and 
Line Q~2] in the same run. This is guaranteed by the fact that 
process p cannot pass the if-conditions in Line[10]or in Line[26l 
after decided p is set to 1, which happens whenever p decides. 

■ 

Lemma 11 (Termination): Every process decides exactly 
once. 

Proof: Lemma [10] shows that every process decides at 
most once. We will now show that every process decides at 
least once. First, we will show that there is a root component 
in every round. Consider the strongly connected components 
that partition the set of nodes of the stable skeleton graph Q nr 
in some round r. Such a set always exists, since the strongly 
connected components form equivalence classes of nodes. It 
is well known that the contraction of the strongly connected 
components is a directed acyclic graph, which reveals that 
there is at least one node C in the contracted graph that has 
no incoming edges. Clearly, C satisfies the definition of a 
root component in Q nr . Therefore, there is a nonempty set 
R r of strongly connected components all of which are root 
components in round r. 

Let r ^ 1 be the earliest round where Q nr is stable for at 
least n — 1 rounds, i.e., Vr' G [r, r + n — 1] : Q n r = Q nr . 
Note that property ([Tj implies that r exists. Now, consider any 
root component lZ r G R r : Clearly, since every process is in 
exactly one strongly connected component, we have 

Vp G K r : C r „ = ll r = r R, r+n ~ 1 = C; +n -\ (11) 



We will now show that the approximated skeleton graph of 
such a process p is in fact exactly the strongly connected 
component of p. Consider anyp G TZ r (= Cp + ™~ 1 ). First, since 
(r + n-1) ^ n, Lemma|5]and CO) imply that K r C GJ+™- 1 . 
We will now show that 1Z T 2 GI +n_1 , which proves that these 
graphs are equal: Since G r p +n ~ is connected by construction, 
it is sufficient to show that every edge in G p +n ^ 1 is also in TV. 

Assume in contradiction that there is an edge e = (<?' — > q) 
in Gp + " _1 such that q e TZ r but g' £ ft r ; note that the 
other way round (q 1 G 7?. r but q ^ 7£ r ) is impossible by 
construction. Using Lemma [6] we know that q' G PT(q,r'), 
and Observation Q] implies that r' > (r + n — 1) — n = r — 1, 
i.e., r' ^ r. Then, by definition, we have that e G G nr , i.e., 
e is an incoming edge of 7?. r , contradicting the assumption 
that TV is a root component. We can therefore conclude that 

By assumption, TV is a root component, which tells us that 
Qr+n-i j s singly connected, i.e., p will pass the if-condition 
in Line [28] in round r + n — 1 and decide. Recall the contracted 
stable skeleton graph of round r + n — 1. Since every path in 
this graph is rooted at some node corresponding to a root 
component in the set R r . Thus, all processes that are not in 
a root component will receive a decision message by round 
r + 2n — 1 and also decide, which completes our proof. ■ 

In the remainder of this section we will prove that Algo- 
rithm [U satisfies the k-agreement property. We will start out 
with some basic invariants on decision estimates. 

Observation 2 (Monotonicity): In any run of Algorithm Q] 
it holds that Vr > 0: a£ > x r p +1 . 

Lemma 12: If process p does not decide in Line [12] we 
have that Vr ^ n — 1 : x p = x p +l . 

Proof: Suppose that there is an r ^ n— 1 such that p sets 
Xp +1 •(— x q and x r p ^ x q . This can only occur in Line [27] if the 
process does not decide in Line Q~2] From Observation |2] and 
validity (cf. Lemma |9), we know that p did not previously 
receive x q and that x q is the initial value of some distinct 
process q. Since processes forward their estimated decision 
value in every round, ([3} implies that the shortest path from 
q to p (along which x p has been propagated to p) in Q nr+1 
has length r + 1. However, this is impossible as r + 1 ^ n 
and the longest possible path has length n—1. ■ 

Lemma 13: Suppose that some process p decides on x p in 
round r by executing line [T2] Then some process q ^ p has 
decided on x p in round r' < r by executing Line [29] 

Proof: Every process decides either in Line [29] or in 
Line[T2] but not both ( Lemma ITOb. Since p decided in Line[T2l 
it must have received a (decide, x q ,_) message from some 
distinct process q. If q decided in Line [29] we are done; 
otherwise q decided in Line [12] in round i — 1, we can repeat 
the same argument for q. After at most n—1 iterations, we 
arrive at some process that must have decided using Line [29] 

■ 

Lemma 14: Let C p be the strongly connected component of 
process p in round n. Then, it holds that \/q G C p : x™ = x p . 



Proof: First, observe that due to Lemma [13] and the fact 
that no process can pass the check in Line [28] before round n, 
no process can decide before round n. Therefore, processes 
can update their estimate values until at least round n. 

Suppose that there are processes p, q G C p , such that x p ^ 
x q . In particular we assume without loss of generality, that x q 
is minimal among all round n estimation values of processes 
in C r p , i.e., x™ > a;™. 

Let r q be the round where q first sets x q to the value 
x q . By Observation [2] it follows that q does not update x q 
anymore before round n. Since Algorithm Q] satisfies validity 
(Lemma [9), we know that there is some process s that is 
the source of this value, i.e., s initially proposed x q . By the 
code of the algorithm we know that in round r process p only 
considers values in Line [27] that were sent by some process 
in PT(p,r). This implies that there is a sequence of pairwise 
distinct processes s = q\, . . . , qt = q, such that 

Vi,(l^i<e):q t ePT(q t+1 ,i). (12) 

Clearly, r q = I — 1. Let j ^ I be such that qj G C p and j is 
minimal, let T q be the path in Q n 1 induced by the sequence 
s up to qj. Moreover, since qj G C p , there is a path T p in C p 
from qj to p. Since C p C Q n 1 , T p is a path in Q n 1 as well. 
Let T be the path in Q n 1 obtained by appending T p to T q . By 
construction T is simple, and therefore its length is bounded 
by n—1. Moreover, the initial value of s was propagated along 
this path — over T q by construction and over T p , because x q is 
minimal in C p . This leads to process p assigning this value to 
x p in some round r p ^ which contradicts the assumption 
that x r p L >x n q . ■ 

Lemma 15 (k- Agreement): Processes decide on at most k 
distinct values. 

Proof: For the sake of a contradiction, assume that there is 
a set of £ > k processes D = {pi, . . . ,pg} in a run a where 
Pi decides on x°° = x^ in round r; ^ n and \fpi,pj G 
D : ^ x^. By virtue of Lemma [T3~l we can assume that 
every pi has decided by executing Line [29] Considering that 
no process decides before round n, applying Lemma [T2l yields 
that 

Vr>nVp hPj eD:x r pi ^x r p .. (13) 

Note that the approximated skeleton graphs G^* and Gp] are 
strongly connected in round ri resp. r 3 , otherwise the pro- 
cesses could not have passed the if-condition before Line [29] 
We will first show that the different decision values of P i 
and pj imply that their approximated skeleton graphs in rounds 
ri resp. rj are disjoint. Lemma [7] reveals that these skeleton 
graphs are contained within the respective strongly connected 
components of an earlier round, i.e., 

Crj-n+l 3 G n and D (/; . 

If these strongly connected components of pi and pj are 
disjoint, then so are the approximated skeleton graphs and 

4 Note that x^f denotes p's final "estimate", i.e., the actual decision value 
of process p. 



we are done. Therefore, assume in contradiction that 

i = c r p '- n+1 nc r p f n+1 

We will now prove that one of these components contains 
the other. Without loss of generality, suppose that ^ rj and 
consider any node p G / C C r p ] n+ . Clearly, p is strongly 
connected to every node in Cp] n+ . Let Z be the induced 
subgraph of Cp] ™ +1 in the skeleton graph g nr i- n + 1 , gy the 
subgraph property (|5) and since r,i ^ rj, it follows that Z = 
Cpf n+1 , and hence ZnC£-™ +1 ^ 0. By the fact that pel, 
we know that p € CI? - ™" 1 " 1 . That is, in the skeleton graph 
gnn-n+i^ p rocess p j s strongly connected to all nodes in 
Cpi~ n+1 and Z. But since the strongly connected component 
Cpi~ n+1 is maximal, we actually have 

QTi—n+l — ^ ^ Qrj—n-\-l 

which means that pj G C r p i ~ n+1 . Then, Lemma [141 readily 
implies that \/q G C p i~ n+1 it holds that = x™ and, in 
particular, ir™ = a;™^, which contradicts (1131 . We can there- 
fore conclude that the intersection of the strongly connected 
components, and therefore, by Lemma [7] also the intersection 
of Gp and G v 3 - is indeed empty, i.e., 

Vp*,Pje£>:(G2nGg) = 0. (14) 

By Theorem|8]it follows that each of the strongly connected 
approximated skeleton graphs GI* can be partitioned into a set 
Di of strongly connected components in Q n 00 . By Theorem[T| 
at most k of the sets D, can contain a root component. Note 
that (TPfl i implies that no strongly connected component is 
in two distinct sets Di, Dj. For the sake of a contradiction, 
assume that (w.l.o.g.) the set Di corresponding to G p e e does 
not contain a root component. Now consider the contracted 
graph of Q n °° where the nodes are the strongly connected 
components. Since the contracted graph is acyclic, it follows 
that there exists a path V in the (non-contracted) graph Q n 00 
that ends at process pi G Di, and is rooted at some process 
q G where is a root component and thus by assumption 
not in Dp. However, by the subgraph property dl}, we know 
that the path T is also in Q nre . But then Lemma [4] implies that 
q G Gp% and Theorem [8] shows that G D(, i.e., one of the 
components in D( in fact is a root component. This provides 
the required contradiction. ■ 

Theorem 16: Algorithm Q] solves fc-set agreement in system 
P«»(A). 

Proof: Lemma Q3] implies k-agreement. Termination is 
guaranteed by Lemma QT| and Lemma [9] shows that validity 
holds. ■ 

V. Discussion and Future Work 

We have introduced the notion of communication graphs 
and presented an algorithm that approximates the stable skele- 
ton of a run. The algorithm is based on exchanging local 
approximations of the stable skeleton, hence has a worst-case 
message bit complexity that is polynomially in n. We have 
also introduced a class of communication predicates V MC s(k) 



and proved that using this approximation one can solve fc- 
set agreement in a system that guarantees 'Psrcs(fc)- Note that 
the algorithm actually solves consensus in sufficiently well- 
behaved runs. 

The one-to-one correspondence between the (at most) fc root 
components of the stable skeleton graph and distinct decision 
values shows that these communication graphs are a promising 
new tool for studying the underlying synchrony in a system. 
Since our algorithm yields a correct approximation atop of 
any communication predicate, part of our future work will 
be devoted to finding a graph-theoretic characterization of the 
weakest synchrony requirements for different agreement prob- 
lems and further exploring the duality between communication 
predicates and graph-theoretic properties. 
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